1. Field of the Invention
The present invention relates to a method and a system of database divisional management for use with a parallel database system. More particularly, the invention relates to a database divisional management method and a parallel database system whereby the number of processors or the number of disk units for database processing is optimized under given loads.
2. Description of the Prior Art
Parallel database systems are proposed illustratively by David Dewitt and Jim Gray in xe2x80x9cParallel Database Systems: The Future of High Performance Database Systemsxe2x80x9d (CACM, Vol. 35, No. 6, 1992, pp. 85-98). Parallel database systems of the kind proposed above involve having a plurality of processors tightly or loosely connected to one another and subjected to database divisional management.
How to configure a conventional parallel database system is at the user""s discretion. Once established, the conventional system configuration must remain as it is. With its lack of flexibility, the system configuration may be unsuitable from the start for dealing with envisaged loads or may become incapable of addressing the loads some time after the start of its operative state. In such cases, the desired degree of parallel processing is not available and/or high-speed query processing is not implemented.
It is an object of the present invention to provide a database divisional management method and a parallel database system whereby desired degrees of parallel processing and high-speed query processing are available.
In carrying out the invention and according to a first aspect thereof, there is provided a database divisional management system for use with a parallel database system having a storage medium, storage and management means for storing and managing a database in the storage medium, and a plurality of access means for accessing the database in response to query inputs. This database divisional management system comprises: generation means for generating a procedure for processing the query inputs; division means for dividing the database into a plurality of partitions in accordance with the load pattern provided for executing the generated processing procedure; and determination means for determining the number of access means for simultaneously accessing the partitions of the database.
In a preferred structure according to the invention, the storage and management means determines the physical addresses corresponding to logical addresses at which the plurality of access means access the partitions of the database.
With this structure, the load pattern is determined by the access efficiency of each of the access means and by the amount of information stored in the partitions of the database accessed by the access means.
According to a second aspect of the invention, there is provided a database divisional management method for use with a parallel database system comprising an FES node for analyzing and optimizing user queries and generating a processing procedure in response thereto, BES nodes for accessing a database on the basis of the processing procedure generated by the FES node, an IOS node having a storage medium (i.e., disk units) and capable of storing and managing the database in the storage medium, and a network for connecting the FES, BES and IOS nodes. This database divisional management method comprises the steps of: calculating the load pattern by which to perform database processing using the processing procedure; and determining the number of processors assigned to the FES node, the number of processors assigned to the BES nodes, the number of processors assigned to the IOS node, the number of disk units of the IOS node, and the number of partitions of the disk units in accordance with the load pattern for data processing.
According to a third aspect of the invention, there is provided a database divisional management method for use with a parallel database system comprising an FES node for analyzing and optimizing user queries and generating a processing procedure in response thereto, BES nodes having a storage medium (i.e., disk units) in which to store a database and capable of accessing the database on the basis of the processing procedure generated by the FES node, and a network for connecting the FES and BES nodes. This database divisional management method comprises the steps of: calculating the load pattern by which to perform database processing using the processing procedure; and determining the number of processors assigned to the FES node, the number of processors assigned to the BES nodes, the number of disk units of the BES nodes, and the number of partitions of the disk units in accordance with the load pattern for database processing.
According to a fourth aspect of the invention, there is provided a database divisional management method for use with a parallel database system comprising an FES node for analyzing and optimizing user queries and generating a processing procedure in response thereto, BES nodes for accessing a database on the basis of the processing procedure generated by the FES node, an IOS node having a storage medium (i.e., disk units) and capable of storing and managing the database in the storage medium, and a network for connecting the FES, BES and IOS nodes. This database divisional management method comprises the steps of: determining the upper limit number of pages which are accessible in parallel and which require a constant time each when the database is scanned for access thereto; and determining the number of processors assigned to the FES node, the number of processors assigned to the BES nodes, the number of processors assigned to the IOS node, the number of disk units of the IOS node, and the number of partitions of the disk units in accordance with the upper limit number of pages.
According to a fifth aspect of the invention, there is provided a database divisional management method for use with a parallel database system comprising an FES node for analyzing and optimizing user queries and generating a processing procedure in response thereto, BES nodes having a storage medium (i.e., disk units) in which to store and manage a database and capable of accessing the database on the basis of the processing procedure generated by the FES node, and a network for connecting the FES and BES nodes. This database divisional management method comprises the steps of: determining the upper limit number of pages which are accessible in parallel and which require a constant time each when the database is scanned for access thereto; and determining the number of processors assigned to the FES node, the number of processors assigned to the BES nodes, the number of disk units of the BES nodes, and the number of partitions of the disk units in accordance with the upper limit number of pages.
According to a sixth aspect of the invention, there is provided a database divisional management method for use with a parallel database system comprising an FES node for analyzing and optimizing user queries and generating a processing procedure in response thereto, BES nodes for accessing a database on the basis of the processing procedure generated by the FES node, an IOS node having a storage medium (i.e., disk units) and capable of storing and managing the database in the storage medium, and a network for connecting the FES, BES and IOS nodes. This database divisional management method comprises the steps of: calculating the expected degree of parallelism p according to the load pattern based on the processing procedure; and determining the number of processors assigned to the FES node, the number of processors assigned to the BES nodes, the number of processors assigned to the IOS node, the number of disk units of the IOS node, and the number of partitions of the disk units in accordance with the expected degree of parallelism p.
According to a seventh aspect of the invention, there is provided a database divisional management method for use with a parallel database system comprising an FES node for analyzing and optimizing user queries and generating a processing procedure in response thereto, BES nodes having a storage medium (i.e., disk units) in which to store and manage a database and capable of accessing the database on the basis of the processing procedure generated by the FES node, and a network for connecting the FES and BES nodes. This database divisional management method comprises the steps of: calculating the expected degree of parallelism p according to the load pattern based on the processing procedure; and determining the number of processors assigned to the FES node, the number of processors assigned to the BES nodes, the number of disk units of the BES nodes, and the number of partitions of the disk units in accordance with the expected degree of parallelism p.
In another preferred structure according to the invention, the database divisional management method further comprises the steps of: calculating the optimum number of accessible pages m; calculating the number of pages s (=m/p) in units of sub-key ranges if key range partitions exist; and having sub-key range partitions in units of s pages for inserting data into a disk apparatus.
According to an eighth aspect of the invention, there is provided a database divisional management method for use with a parallel database system comprising an FES node for analyzing and optimizing user queries and generating a processing procedure in response thereto, BES nodes for accessing a database on the basis of the processing procedure generated by the FES node, an IOS node having a storage medium (i.e., disk units) and capable of storing and managing the database in the storage medium, and a network for connecting the FES, BES and IOS nodes. This database divisional management method comprises the steps of: detecting a load unbalance on the basis of such load information items as the number of accessed pages, the number of hit rows and the number of communications acquired during execution of the processing procedure; and changing the number of processors assigned to the FES node, the number of processors assigned to the BES nodes, the number of processors assigned to the IOS node, and the number of disk units of the IOS node so as to eliminate the load unbalance.
According to a ninth aspect of the invention, there is provided a database divisional management method for use with a parallel database system comprising an FES node for analyzing and optimizing user queries and generating a processing procedure in response thereto, BES nodes having a storage medium (i.e., disk units) in which to store and manage a database and capable of accessing the database on the basis of the processing procedure generated by the FES node, and a network for connecting the FES and BES nodes. The database divisional management method comprises the steps of: detecting a load unbalance on the basis of such load information items as the number of accessed pages, the number of hit rows and the number of communications acquired during execution of the processing procedure; and changing the number of processors assigned to the FES node, the number of processors assigned to the BES nodes, and the number of disk units of the BES nodes so as to eliminate the load unbalance.
In a further preferred structure according to the invention, the database divisional management method further comprises the steps of: closing, when online processing is in progress, the key range of a database table if at least one of the three numbers consisting of the number of processors assigned to the BES nodes, the number of processors assigned to the IOS node and the number of disk units is to be increased, the database table being the object to be managed by either the processors or the disk units to be added; assigning the processors and the disk units anew; succeeding lock information and directory information; updating the dictionary information necessary for node assignment control; and releasing the closing of the key range thereafter if the online processing is still in progress.
In an even further preferred structure according to the invention, the database divisional management method further comprises the steps of: closing, when online processing is in progress, the key range of a database table if either the number of processors assigned to the BES nodes or the number of disk units is to be increased, the database table being the object to be managed by either the processors or the disk units to be added; assigning either the processors or the disk units anew; succeeding lock information and directory information; updating the dictionary information necessary for node assignment control; moving data from the existing group of disk units to the newly added disk units; and releasing the closing of the key range thereafter if the online processing is still in progress.
In a still further preferred structure according to the invention, the database divisional management method comprises the steps of: closing, when online processing is in progress, the key range of a database table if at least one of the three numbers consisting of the number of processors assigned to the BES nodes, the number of processors assigned to the IOS node and the number of disk units is to be decreased, the database table being managed by either the processors or the disk units to be removed; determining either the processors or the disk units to be removed; succeeding lock information and directory information; updating the dictionary information necessary for node assignment control; and releasing the closing of the key range thereafter if the online processing is still in progress.
In another preferred structure according to the invention, the database divisional management method further comprises the steps of: closing, when online processing is in progress, the key range of a database table if at least either the number of processors assigned to the BES nodes or the number of disk units is to be decreased, the database table being managed by either the processors or the disk units to be removed; determining either the processors or the disk units to be removed; succeeding lock information and directory information; updating the dictionary information necessary for node assignment control; moving data from the disk units to be removed to the disk units succeeding those units to be removed; and releasing the closing of the key range thereafter if the online processing is still in progress.
In a further preferred structure according to the invention, either the number of processors or the number of disk units for database processing is changed dynamically.
In operation, the invention of the aspects and preferred structures outlined above provides the following major functions and features:
The database divisional management method according to the second aspect of the invention determines the number of processors assigned to each of the configured nodes, the number of disk units, and the number of partitions of the disk units in accordance with the load pattern for database processing (single item search, single item update, data retrieval, etc.). The invention embodied in this structure provides a system configuration suitable for dealing with the load in question, offers the expected degree of parallelism and permits high-speed query processing.
The database divisional management method according to the fourth aspect of the invention determines the number of processors assigned to each of the configured nodes, the number of disk units, and the number of partitions of the disk units in accordance with the upper limit number of pages which are accessible in parallel and which require a constant time each when the database is scanned for access thereto. The invention embodied in this structure realizes high-speed query processing.
The database divisional management method according to the sixth aspect of the invention determines the number of processors assigned to each of the configured nodes, the number of disk units, and the number of partitions of the disk units in accordance with the expected degree of parallelism p according to the load pattern. The invention embodied in this structure provides the expected degree of parallelism.
The database divisional management method of a preferred structure calculates the number of pages s in units of sub-key ranges using the optimum number of accessible pages m and the expected degree of parallelism p (s=m/p), and gets sub-key range partitions in units of s pages for inserting data into a disk apparatus. The invention embodied in this alternative structure allows data to be managed in substantially equal partitions.
The database divisional management method according to the eighth aspect of the invention detects a load unbalance, and changes the number of processors assigned to each of the configured nodes or the number of disk units so as to eliminate the detected load unbalance. The invention embodied in this structure keeps the system always configured to allow for the load fluctuation, provides the expected degree of parallelism and permits high-speed query processing.
When online processing is in progress, the database divisional management method of another alternative structure closes the key range of the relevant database table if either the number of processors assigned to each of the configured nodes or the number of disk units is to be increased or decreased. After this, necessary information is succeeded and the closing of the key range is released. This structure minimizes the overhead and, in a system configuration comprising the IOS node, allows the data to be succeeded without moving them across the disk units.
The database divisional management method of yet another preferred structure allows either the number of processors or the number of disk units for database processing to be changed dynamically. This feature readily provides for a scalable parallel database system.
The foregoing and other objects, advantages, manner of operation and novel features of the present invention will be understood from the following detailed description when read in connection with the accompanying drawings.